/IDA  043240 


1 


Professional 

Paper 

2-77 


HumflHO‘PP-2-77 


APOSR.TR*  77-* 0 9 9 0 


i \ 


j # 

u 


HumRRO ' 

a? 


Some  Current  Problems  in 
Simulator  Design,  Testing  and  Use 


Paul  W.  Caro 


SEVILLE  RE8EARCH  CORPORATION 

400  Plus  Building,  Panucola,  Florida  32S05 


D D C 


UK  **'  wn 


Papar  prtstntad  at  tha  38th  Military  Oparatiom 
Ratearch  8ympotlum,  Fort  Euttii,  Virginia; 

7-9  Daeambar  1976 


lEistsinns 

CK 


HUMAN  RESOURCES  RESEARCH  ORGANIZATION 

300  North  Washington  Street  • Alexandria.  Virginia  22314 


Approvad  for  public  ralaata;  distribution  unlimitad. 


Praparad  for  . j 

Air  Pores  Office  of  Scientific  Reessreh  , j 

Air  Pores  Systems  Command 

Boiling  Air  Pores  Bess,  O.C.  20332  | 

a 1 


Approvsd  for  ruMlo  vnlo aaej 
llst.rl  button  unllalt.ai. 


-»  

'UCURlTy  CL  ASSIFIC  ATION  or  This  PAT.  F.  (H’h.ri  l»la  Tnlr-M) 


EPORT  DOCUMENTATION  PAGE 


Rt-.AP  INSTKWC  riONS 
IlKI-ORK  fUMIM.KTINc;  I- ( 1|«'M 


ir  NMtlPllNT'S  CATA'.nc,  NIJMHFB 


VI, 


PERFORMING  0 RG  ANI  2 AT  ION  NAME  AND  ADDRESS 

Human  Resources  Research  Organization 
300  North  Washington  Street 
Alexandria,  Virginia  233  14 


M.  CONTROLLING  OFFICE  NaME  AMO  ADDRESS 


F446^d/-76-C-tm8 


j;L1A:mHB3TrPPiJ-f.lin3 


AREA  A WORK  UNIT  NUMBERS 


6 1 1 02  F 
2313/A2 


..  mhmmu  date.  . 


Air  Force  Office  of  Scientific  Research  {NL){/>',  Mar(9i  W77 
Bolling  A FB  DC  20332  ~ 


15.  SECURITY  CLASS,  (ol  thf  r«j tort) 

Unclassified 


I5«  DELL  ASSIFIC  ATION'  DOWNGRADING 
SCHEDULE 


I*  DISTRIBUTION  STATEMENT  fot  (hl» 


Approved  for  public  release;  distribution  unlimited. 


19.  KEY  WORDS  fCoM/Imm  on  ulrte  If  ni*r  **<t*fv  am/  #rf»nfh'i-  by  block  number) 


Simulation  (lost  K f Tee  t i voiles;; 

Aircraft  Simulators  Instructor  Training 

Training  lievlees  Simulator  Design 

Training  Kf fee t I veness  Simulator  Training 

Training  Models __ 

20  ABSTRACT  (Cnnflmm  nn  r«v»r.««  Mil*  If  w*r#»m#rv  Jthtf  Itfenfffv  hy  Nock  number) 

■''This  report  Is  eonecriuni  with  the  general  problem  of  the  ef  fee  t I veness  of 
simulator  training,  ami  re  fleet.::  Information  developed  during  the  conduct  of 
aircraft,  simulator  training  research  projects  sponsored  by  the  Air  Foret*, 
Army.  Navv,  and  Coast  Cuard.  Problems  are  identified  related  to  simulator 
design,  tost  lug  and  use  that  impart  simulator  training  ef feet [veness.  These 
problems  are  (I)  isolation  of  the  simulator  user  from  the  design  and  develop- 
ment process,  (2)  I nuL tent 4 on  to  behavioral  and  training  models  during  that 
process,  ( I ) Ignoring  training  considerations  during  simulator  testing, 


stCURITV  CLASSIFICATION  OF  THIS  PAGE  (’H'hi.ri  nm*  Kiilrr(>rl)  , _______________________ 

(Report  documentation  page  ' 

A ocnfiai-^ijullM . I-./H/T  ACCE.'ION  no.,!*'  RECIPIENT'S  catalog  number 

) AFQSmTR-  V 7 - H J l s 


4.  TITLE  (mndSubtnirl 


SOME  CURRENT  ^PROBLEMS  IN  SIMULATOR  ( ! Interim  Scientific  Ke^S, 


‘DESIGN,  TESTING  AND  USE  , 


S T fPt-OF'REPORTS'PERlOO  COVERED 


-fri  ntnronwiffTf-OROr^wgPQPT'l(( u m h i r •' 

I HumRRO-PP-2-77  ( ~ 


AU  THORf-O 


Paul  W.  /Ca  ro 


■ AS  ■ F446^f-76-C-/0118 


9 PERFORMING  ORGANIZATION  name  ANO  aodress 

Human  Resources  Research  Organization 
300  North  Washington  Street  ! 

Alexandria,  Virginia  22314 /_ 

II.  CONTROLLING  OFFICE  N«ME  and  AODRESS  i'll 

Air  Force  Office  of  Scientific  Research  (NL)//7/ 
Bolling  A FB  DC  20332 


to.  PROGRAM  ELEMENT.  PROJECT.  T ASK 
AREA  6 WORK  UNIT  NUMBERS 

61102F 

2313/A2 

] V2^-«E  R O FU_  Q.AXE . 

/ Mar  & $77  | 

i rr  n uwh  rft'crrrnnrt? 


14.  MONITORING  AGENCY  NAME  a AODRESSfff  dlfierent  from  Controlling  Office)  15.  SECURITY  CLASS,  (of  thle  report) 

/ y ( Unclassified 


I5».  DECLASSI  FI  CATION/ DOWN  GRADING 
SCHEDULE 


16.  DISTRIBUTION  STATEMENT  (o I thl*  Report) 


Approved  for  public  release;  distribution  unlimited. 

I & Tl 


I 17.  OISTRIBUTION-SXATEIAEMT  (of  in*  tr>»irK7Tenlere"3Tn~ 


(&)  \ v-V- 


/ / ■'> 


rr  r 


18.  SUPPLEMENTARY  nc 


l 10.  KEY  WORDS  (Continue  on  .......  .ride  II  nereeeery  end  Identity  hy  block  number) 


Simulation  Cost  Kffeot i vencss 

Aircraft  Simulators  instructor  Training 

Training,  Devices  Simulator  Design 

Training  Fffect i veness  Simulator  Training 

Train  11))’.  Models 

20  ABSTRACT  (Continue  on  rover..*  el  do  II  neceeenry  etui  Identity  hy  btnrk  number) 

-x  Tills  report  is  concerned  with  the  general  problem  of  the  effectiveness  of 
simulator  training  and  reflects  information  developed  during  the  conduct  of 
aircraft  simulator  training  research  projects  sponsored  hy  the  Air  Force, 

Army,  Navy,  and  Coast  Guard.  Problems  arc  identified  related  to  simulator 
design,  testing,  and  use  that  impact  simulator  training  e f fee  t i veness . Those 
problems  arc  (1)  isolation  of  the  simulator  user  from  the  design  and  develop- 
ment process,  (2)  inattention  to  behavioral  and  training  models  during  that 
proeess,  (!)  ignoring,  training  considerations  during,  simulator  testing, 


DD  . \Zr  1473  rniTirm  or  i nov  as  is  obsolete 


Unci  as  si  fieri  ^ 0 ....  ( ( 


Unclassified 


ECUKlTy  Cl  OSSIFICATION  OF  THIS  PAGtfWim  C»l«  F.nlitttl) 


20 . (ront . } 

(4)  inadequate  feedback  to  simulator  designers  concerning  simulator  training 
effectiveness,  (5)  inattention  to  techniques  of  simulator  training  that  differ 
from  techniques  of  aircraft  training,  (6)  inadaquate  training  for  simulator 
instructors,  (7)  use  of  rate  of  simulator  utilization  as  an  index  of  its  train- 
ing effectiveness,  and  (8)  Inadequacies  of  simulator  training  cost  effective- 
ness data.  . 


PREFATORY  NOTE 


This  paper  is  based  upon  a presentation  made  by  the  author  at  the 
38th  Military  Operations  Research  Symposium  in  December,  1976.  The  paper 
is  concerned  with  the  general  problem  of  the  effectiveness  of  simulator 
training  and  reflects  information  developed  during  the  conduct  of  aircraft 
simulator  training  research  projects  sponsored  by  the  Army,  Navy,  Air 
Force  and  Coast  Guard.  The  principal  focus  of  the  paper  is  the  identifi- 
cation of  problems  related  to  simulator  design,  testing  and  use  that 
impact  simulator  training  effectiveness. 

The  preparation  of  this  paper  was  supported  by  the  Life  Sciences 
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the  author  was  a Senior  Staff  Scientist  with  the  Human  Resources  Research 
Organization  (HumRRO)  and  was  completed  under  the  auspices  of  Seville 
Research  Corporation  under  HumRRO  Subcontract  No.  SubE77-04~05 . 
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INTRODUCTION 


It  has  become  the  policy  of  the  Federal  government  to  treat  aircraft 
simulators  as  a major  training  resource.  Led  by  examples  set  by  commercial 
airlines  and  spurred  on  by  the  energy  crisis,  our  military  flight  training 
agencies  are  relying  upon  modern  simulators  for  a significant  portion  of  the 
training  of  air  crews.  It  is  becoming  increasingly  difficult  for  even  the 
most  conservative  and  traditional  training  program  manager  to  justify  exten- 
sive use  of  aircraft  for  training  when  simulators  are  available.  It  is  there- 
fore imperative  that  our  military  simulators  be  optimally  designed  and  used. 
Otherwise,  our  state  of  preparedness  could  suffer  disastrously. 

Fortunately,  advancements  in  a number  of  technologies  have  led  to  the 
development  of  simulators  that  are  very  convincing  representations  of  aircraft. 
It  is  now  possible  to  perform  in  simulators  many  of  the  complex  tasks  required 
during  operational  missions,  and  tests  have  shown  that  simulators  can  be  used 
effectively  to  develop  many  of  the  skills  underlying  those  tasks.1  Overall, 
the  simulator  training  available  with  today's  engineering  technology  can  be 
quite  effective,  and  in  many  instances  its  extensive  use  as  a substitute  for 
training  ir.  aircraft  is  probably  well  justified.  The  quality  of  most  simulator 
training  activities  today  is  unquestionably  superior  to  that  of  a decade  ago. 

In  spite  of  the  fact  that  simulators  are  better  than  ever,  there  are 
problems  within  the  overall,  simulator  training  system  that  mitigate  against 
effective  simulator  training.  These  problems  are  technical,  conceptual,  and 
managerial  in  nature,  and  they  encompass  all  aspects  of  simulator  design, 
testing  and  use.  Technical  problems  related  to  simulator  engineering  have 
tended  to  receive  the  most  attention,  but  conceptual  and  managerial  problems 
require,  attention  because  they  also  can  reduce  the  ef fectiveness  of  simulator 
training. 

The  purpose  of  this  paper  is  to  call  attention  to  some  of  the  conceptual 
and  managerial  problems  that  the  writer  has  noted  during  reviews  of  simulator 
projects  of  the  Army,  Navy,  Air  Force  and  Coast  Guard.  Although  there  are 
important  differences  in  the  way  each  service  conceptualizes  and  manages  its 
simulator  programs,  there  are  many  commonalities  to  those  programs,  and  some 


1Caro,  P.W.  "Aircraft  Simulators  and  Pilot  Training,"  Human  Factors,  1973, 
15,  502-509. 
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o£  the  problems  described  below  arc  common  to  all  lour  services.  It  should 

be  noted,  however,  that  the  following  discussion  is  not  directed  at  a par- 
ticular simulator  project,  simulator  procurement  agency,  or  simulator  user. 
Instead,  the  discussion  is  an  attempt  to  synthesize  types  of  problems  that 
can  be  found,  in  varying  degrees,  in  each  of  the  services'  programs. 

In  order  to  provide  a structure  to  the  discussion,  problems  will  be 
identified  in  relation  to  three  phases  of  the  life  cycle  of  a simulator. 

These  are  the  Design  Phase,  the  Testing  Phase,  and  the  Use  Phase.  Two 
problems  associated  with  the  Design  Phase  will  be  described  first. 

SIMULATOR  DESIGN 

Isolation  of  the  Simulator  User. 

Because  of  the  many  specialized  and  technical  functions  associated  with 
simulator  design  and  procurement,  there  has  been  a tendency  for  agencies  res- 
ponsible for  these  functions  to  be  staffed  with  specialists  who  can  perform 
them  in  an  efficient  manner.  Such  agencies  have  full  responsibility  for  de- 
signing simulators  to  meet  the  reported  or  observed  needs  of  the  eventual  user 
of  the  simulators.  Concurrently,  the  eventual  user  is  relieved  of  any  res- 
ponsibility for  simulator  design  and  procurement  so  that  he  may  be  about  his 
business  of  training. 

This  separation  of  responsibilities  between  designer  and  user  during  the 
simulator  design  phase  has  unfortunate  consequences:  (1)  it  places  all  <1  sign 

decisions  in  the  hands  of  specialists  who  may  lack  an  understanding  of  training 
processes,  (2)  and  it  tends  to  isolate  the  user  from  the  design  process  so  that 
information  that  could  be  helpful  to  the  design  specialist  is  never  called  to 
his  attention.  The  resulting  simulators  sometimes  are  different  from  what  the 
user  expected  them  to  be  when  he  submitted  the  original  requirement  statement. 

In  the  absence  of  a strong  user  influence  during  the  simulator  design 
process,  decisions  can  be  made  that  compromise  the  future  device's  training 
potential.  The  user's  needs  are  not  necessarily  the  prime  concern  of  the  de- 
signer. The  effectiveness  of  the  agency  responsible  for  simulate:  design  is 
measured  principally  in  terms  of  its  adherence  to  budget  and  time  constraints, 
not  whether  the  simulator  is  useful  for  training.  This  point  deserves  emphasis. 
Simulator  procurement  agencies  have  no  explicit  responsibility  Tor  simulator 
training  effectiveness.  Training  with  simulators  is  the  user's  responsibility. 
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If  future  simulators  are  to  be  effective  training  tools,  the  user  must  be 
actively  involved  in  the  important  decisions  made  during  the  simulator's 
design  phase,  not  just  represented  by  someone  whose  concerns  may  be  different. 

Inattention  to  Behavioral  and  Training  ModelB. 

Another  problem  associated  with  the  design  phase  has  to  do  with  the  fact 
that  simulators  are  designed  to  simulate  rather  than  to  train.  They  are  de- 
signed to  reflect  physical  models  rather  than  behavioral  or  training  models. 
This  fact  was  reflected  in  a comment  made  at  a recent  conference  on  training 
and  simulation . ^ A participant  in  that  conference  was  overheard  to  complain 
"1  came  here  to  learn  about  simulators,  not  about  training." 

Modern  flight  simulators  arc  complex  physical  systems  that  simulate  other 
complex  physical  systems,  e.g.,  aircraft.  Because  of  this  emphasis  upon  physi- 
cal systems,  it  is  not  surprising  that  they  usually  reflect  physical  rather 
than  behavioral  models.  They  look,  feel,  and  sound  very  much  like  the  physical 
creations  after  which  they  were  modeled,  but  the  relationships  between  the  phy- 
sical features  of  the  simulators  and  the  training  for  which  they  primarily  were 
Intended  Is  not  always  apparent. 

The  principal  physical  models  used  by  flight  simulator  designers  are  the 
aircraft  and  the  flight  environment.  The  aircraft  being  simulated  is  a design 
model  that  determines  the  physical  size  and  appearance  of  the  portions  of  the 
simulator  with  which  the  trainees  interact  (i.e.,  the  trainee  station)  and  the 
manner  in  which  the  controls  and  displays  located  there  function.  Usually  thiB 
model  is  well  defined  through  aircraft  design  and  flight  test  data,  and  the 
dominant  role  of  this  mode]  in  simulator  design  is  apparent. 

The  atmospheric,  electronic  and  visual,  environments  in  which  the  design 
model  aircraft  operates  are  other  models  that  play  major  roles  in  simulator 
design.  Since  these  environment  models  are  easily  quantified,  they  are  readily 
usable,  and  their  precise  representation  in  simulator  design  also  receives  a 
great  deal  of  attention. 

A behavioral  model  of  the  training  process  should  be  equally  important  in 
simulator  design,  but  behavioral  considerations  usually  receive  relatively 
little  attention  from  simulator  designers.  A training  process  model  would 
describe  fo  the  designer  how  the  simulator  will  be  used  f:o  accomplish  the 

^The  9th  Naval  Training  Equipment  Center/Industry  Conference,  Orlando,  FL, 

March  9-11,  197fi. 
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objectives  prescribed  for  it,  but  unfortunately,  a well  defined  model  for 
the  learning  process  has  never  been  developed  for  simulator  training.  Con- 
sequently, simulator  designers  have  been  forced  to  ignore  the  training  process 
on  the  assumption  that  faithful  adherence  to  the  aircraft  and  environmental 
models  will  provide  an  adequate  setting  for  an  instructor  to  conduct  training 
in  a simulator  much  as  he  would  in  an  aircraft. 

During  the  past  decade,  the  adequacy  of  the  in-flight  or  aircraft  train- 
ing process  model  for  simulator  training  has  been  challenged,  and  suggestions 
have  been  made  that  a more  appropriate  training  process  model  would  take  ad- 
vantage of  the  simulator  as  a learning  environment  not  subject  to  some  of  the 
limitations  or  negative  learning  features  of  the  aircraft.  While  such  a con- 
cept has  merit,  a suitable  training  process  model  based  on  the  concept  has  not 
yet  been  defined  and  articulated  for  use  by  simulator  design  engineers. 

Because  of  the  inadequacy  of  the  existing  simulator  design  models  related 
to  the  training  process,  the  simulator  designer  designs  new  simulators  that  are 
very  much  like  the  last  ones  he  designed,  or  at  least  he  looks  to  an  existing 
simulator's  instructor  station  and  instructional  features  as  a model  for  com- 
parable portions  of  his  new  design.  Thus,  many  of  today's  simulators  are  being 
designed  very  much  like  chose  of  decades  ago,  largely  because  no  other  rele- 
vant design  models  are  available.. 

SIMULATOR  TESTING 

Simulator  Tests  Ignore  Training  Suitability. 

Now  let  us  turn  to  the  Testing  Phase  of  the  Simulator  life  cycle.  Here 
again,  there  are  two  problems  that  warrant  attention.  The  first  problem  in- 
volves the  device's  initial  testing,  a process  that  occurs  at  the  completion 
of  the  simulator  development  process.  Emphasis  in  these  tests  is  two-fold; 
first,  upon  the  simulator's  conformity  to  design  hardware  specifications,  and 
second,  upon  simulator  reliability  and  maintainability.  These  two  areas  of 
consideration  are  important,  of  course,  but  where  in  this  testing  process  is 
it  determined  that  the  device  is  appropriate  to  the  training  requirement?  If 
a military  service  buys  an  aircraft,  a missile,  a ship  or  a tank,  it  conducts 
extensive  tests  to  determine  the  equipment's  suitability  to  perform  a military 
mission.  How  often  does  one  hear  of  a mission  suitability  test  for  a simu- 
lator that  emphasizes  mission  accomplishment  (i.e.  , training)  over  hardware 
factors? 
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It  should  be  apparent  that  the  testing  that  must  take  place  to  determine 
the  acceptability  of  a simulator  must  attend  primarily  to  the  extent  to  which 
the  device  is  suitably  designed  and  constructed  with  respect  to  training  func- 
tions and  processes.  It  should  be  emphasized  that  training  functions  are  the 
primary  considerations  in  device  testing.  Whether  the  device  conforms  to  hard- 
ware and  performance  specifications,  nr  is  reliable  and  maintainable,  are  secon- 
dary considerations.  If  it  is  not  useful  for  training,  the  other  considerations 
are  of  no  concern. 

Obviously,  these  secondary  considerations  must  receive  attention,  and  no 
one  V'ould  advocate  procuring  simulators  that  are  unreliable.  The  concern  is 
not  whether  engineering  tests  should  be  conducted  during  acceptance  testing; 
rather,  the  concern  is  that  most  present  day  simulators  have  been  accepted 
without  regard  to  their  suitability  for  training,  obviously  on  the  assumption 
that  the  user  will  solve  any  problems  that  may  arise  during  training.  Unfor- 
tunately, some  of  the  simulators  that  have' been  accepted  In  the  past  were  so 
poorly  suited  to  training  needs  that  they  literally  could  not  be  used.  The 
user's  solution  in  many  cases  has  been  either  to  initiate  requirements  for 
major  modification  to  the  devices,  or  to  ignore  them  altogether. 

Poor  Feedback  to  Simulator  Designer . 

A second  problem  associated  with  the  simulator  Testing  Phase  relates  to 
information  that  is  obtained  concerning  the  training  mission  suitability  of 
simulators,  either  as  a part  of  an  extended  acceptance  or  operational  testing 
process,  or  as  a result  of  use  of  the  device  following  its  acceptance.  While 
the  designer  often  Is  involved  in  the  conduct  of  acceptance  testing  activities 
related  to  matters  of  engineering  concern,  he  is  not  often  involved  in  activities 
related  to  determining  the  device's  suitability  for  training,  if  indeed  there 
are  any  such  activities  during  the  Testing  Phase.  He  should  be,  of  course,  If 
he  is  to  benefit  from  any  information  gained.  Feedback  to  the  designer  about 
the  adequacy  of  his  design  for  training  is  essential. 

This  problem  relates  to  another  of  the  problems  mentioned  earlier  — 
division  of  responsibility  between  the  simulator  designer  and  user.  Not  only 
is  there  a need  to  keep  the  user  in  the  design  process,  there  is  a need  to  keep 
the  designer  aware  of  the  user's  activities  with  the  device  after  it  has  beer 
put  to  use.  Seldom  do  simulator  designers  have  an  opportunity  to  observe 
simulator  training  except  on  a very  casual  basis  while  touring  training  facil- 
ities. They  have  little  opportunity  to  learn  of  deficiencies  in  existing 
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simulators  that  limit  their  training  value,  and  they  have  even  less  oppor- 
tunity to  interact  with  simulator  instructors  to  suggest  alternate  ways  of 
using  existing  simulators  or  to  discuss  alternate  simulator  design  concepts 
that  might  improve  future  training.  Simulator  designers'  limited  budgets 
sometimes  permit  them  to  visit  government  laboratories  and  manufacturers' 
facilities,  and  occasionally  to  attend  professional  and  technical  meetings. 

But  they  seldom  have  money  left  over  to  support  extended  interaction  with 
simulator  users  or  observation  of  or  participation  in  simulator  training  acti- 
vities. Future  simulator  designs  would  benefit  if  the  designers  were  given 
more  feedback  about  the  training  suitability  of  existing  simulators  and  were 
encouraged  to  consider  innovative  simulator  design  approaches  to  training 
problems  w‘ th  which  they  have  become  familiar. 

SIMULATOR  USE 

Inattention  to  Techniques  of  Simulator  Training. 

The  final  group  of  problems  to  be  discussed  are  those  that  occur  during 
the"  Use  Phase,  the  period  during  which  the  simulator  is  used  to  train  personnel 
operationally.  While  there  undoubtedly  are  many  problems  associated  with  the 
use  of  simulators,  four  are  of  particular  concern.  The  first  has  to  do  with 
the  fact  that  no  one  really  knows  much  about  how  to  train  in  simulators.  Very  • 

little  attention  has  been  devoted  to  the  development  of  a technology  of  simu- 
lator training. 

How  do  we  train  in  simulators?  In  the  case  of  aircraft  simulators,  we 
train  in  simulators  pretty  much  like  we  train  in  aircraft.  In  many  respects, 
that  probably  works  out  reasonably  well,  but  one  cannot  but  wonder  if  there 
might  not  be  a better  way.  It  was  noted  earlier  that  good  models  of  the  simu- 
lator instructional  process  trave  "not  be-en  developed.  The  aircraft- is  a very 
poor  learning  environment,  and  the  flight  instructor  must  function  primarily 
as  a safety  pilot.  In-flight  instruction  is  constrained  by  aircraft  fuel  capa- 
city, the  availability  of  navigation  aids  and  gunnery  ranges,  and  a host  of 
safety  restrictions  that  make  tactics  training  quite  unrealistic  in  some  in- 
stances. Although  simulator  training  can  be  freed  of  each  of  these  restric- 
tions, little  effort  is  being  devoted  to  the  development  of  training  profiles  | 

and  techniques  optimized  for  training  in  simulators  — with  many  of  the  safety 
and  time  restrictions  necessary  in  the  aircraft  inappropriately  carried  into 
the  simulator  as  well. 
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It  would  be  inappropriate  to  criticize  simulator  instructors  for  their 
less  than  optimum  uses  of  simulators.  They  are  doing  exactly  what  they  have 
been  told  to  do  — conducting  Might  training  in  simulators.  If  they  are  to 
do  otherwise,  techniques  of  instruction  more  appropriate  for  simulators  must 
be  developed  for  their  use. 

Inadequate  Training  for  Simulator  Instructors. 

The  second  problem  associated  with  the  use  of  simulators  is  related  to 
the  first.  Even  with  our  present  limited  understanding  of  how  to  use  simu- 
lators, tlie  training  provided  simulator  instructors  is  inadequate.  Simulator 
instructor  training  varies  from  program  to  program,  but  seldom  can  one  find  a 
simulator  instructor  training  program  that  could  be  considered  exemplary.  The 
training  provided  simulator  instructors  varies  from  a "checkout"  on  the  in- 
structor's console  during  which  the  location  and  function  of  switches  and  other 
controls  are  demonstrated  and  questions  are  answered  by  an  already  qualified 
instructor,  to  a more  lengthy  "practice  teaching"  program  in  which  a newly 
selected  instructor  may  spend  a week  or  more  observing  simulator  training  and 
actually  conducting  training  under  the  supervision  and  with  the  assistance  of 
a qualified  instructor.  In  addition,  since  simulator  instructor  trainees  are 
usually  flight  qualified  and  often  instruct  in  aircraft  as  well  as  in  the  simu- 
lator, instructor  training  typically  includes  an  extended  effort  to  raise  their 
proficiency  and  to  standardize  their  performance  in  the  aircraft  itself,  and 
in  some  instances,  such  training  includes  Instruction  in  the  techniques  of 
flight  training,  lesson  planning,  and  preparation  of  instructional  material. 

The  principal  deficiency  of  these  courses,  however,  is  that  they  do  little  to 
provide  the  instructor  the  specialized  knowledge  and  techniques  that  will  allow 
him  to  employ  the  simulator  effe.ctively  and  efficiently  as  a training  vehicle 
that  has  unique  capabilities  different  from  those  of  the  aircraft. 

In  a recent  survey  of  simulator  training,  Frequent  instances  were  found 
in  which  instructors  did  nut  know  how  or  when  to  use  advanced  simulator  train- 
ing features  such  as  adaptive  training,  performance  playback,  and  automatic 
performance  monitoring.^  It  is  indeed  rare  to  find  a simulator  Instructor 
who  has  been  taught  to  use  instructional  techniques  such  as  discovery  learning, 
to  shape  and  reinforce  desired  behaviors,  or  to  employ  individualized  proficiency 


Isley,  R.N. , and  Miller,  E.J.  The  Role  of  Automated  Training  In  Future  Army 
Flight  Simulators.  Final  Report  FK-ED-76-2 7 , Human  Resources  Research  Organi- 
zation, Alexandria,  VA,  October  1976. 
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advancement  as  a training  process  management  tool. 

Documents  available  to  simulator  instructors  related  to  the  use  of  simu- 
lators as  instructional  tools  provide  very  little  how-to-train  guidance.  Typi- 
cally, these  manuals,  handbooks  and  course  outlines  Indicate  the  tasks  to  be 
performed  by  the  trainee  during  a simulator  mission,  but  they  contain  little 
or  no  information  as  to  how  training  is  to  be  accomplished,  how  student  per- 
formance will  be  graded,  and  what  performance  criteria  apply  to  the  trainees, 
given  their  particular  skill  and  experience  levels. 

Emphasis  Upon  Rate  of  Simulator  Utilization . 

A third  problem  associated  with  the  use  of  simulators  is  strictly  a mana- 
gerial problem,  although  its  effects  are  to  reduce  simulator  training  effec- 
tiveness. The  problem  is  simply  that  simulator  training  typically  is  evaluated 
in  terms  of  rate  of  device  utilization,  rather  than  in  terms  of  training  bene- 
fits or  trainee  performance  increments. 

Emphasis  in  many  pilot  training  organizations  at  the  present  time  is  upon 
achieving  high  rates  of  simulator  utilization,  and  quotas,  such  as  80  hours  per 
week  per  cockpit,  are  being  assigned  to  training  units  on  an  apparently  arbi- 
trary basis.  While  the  reasons  for  such  an  approach  may  be  laudable,  the  con- 
sequences are  often  undesirable.  One  such  consequence  is  an  inflated  estimate 
of  the  need  for  simulator  training.  More  importantly,  however,  the  requirement 
that  a simulator  be  used  for  a specified  number  of  hours  per  week  Increases 
the  likelihood  that  at  least  some  simulator  training  will  be  ineffective.  Since 
such  a requirement  is  essentially  unrelated  to  training  needs  or  to  number  of 
personnel  to  be  trained,  keeping  the  simulator  busy  sometimes  involves  repeating 
training  missions  or  exercises  that  already  have  been  mastered.  At  one  loca- 
tion recently  visited  by  the  writer  where  a fixed  rate  of  simulator  utilization 
was  a requirement,  time  recorded  as  simulator  training  time  was  not  always  being 
spent  in  the  device,  and  pilots  of  junior  ranks  were  required  to  use  up  the 
excess  simulator  time  although  they  were  already  proficient  at  tasks  that  could 
be  practiced  in  the  device.  The  negative  effects  of  such  a requirement  upon 
attitudes  toward  simulator  training  are  predictable. 

Inadequate  Simulator  Coat  Effectiveness  Data . 

The  fourth  problem  also  concerns  management  practices.  It  is  simply  that 
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no  one  knows  the  extent  to  which  particular  simulator  training  programs  may 
he  cost  effective.  There  are  two  reasons  for  this.  First,  cost  accounting 
procedures  commonly  employed  by  the  military  services  make  it  difficult  to 
identify  all  relevant  costs  and  to  attribute  them  differentially  to  simu- 
lator versus  aircraft  training.  While  some  of  the  costs  of  operating  and 
maintaining  simulators  can  be  compared  with  similar  costs  associated  with 
aircraft,  the  full  cost  of  each  kind  of  training  is  seldom  known.  Most  train- 
ing managers  would  be  surprised  at  just  how  inexpensive  simulator  training  is, 
compared  to  aircraft  training,  if  the  cost  data  available  to  them  included  the 
full  costs  of  factors  such  as  constructing  and  maintaining  airfields,  gunnery 
ranges,  and  navigational  facilities  used  for  no  purpose  other  than  training. 
Further,  it  is  doubtful  that  most  planners  and  managers  know  how  little  is 
saved  when  a large  number  of  hours  of  aircraft  training  is  shifted  to  simu- 
lators while  all  these  expensive  training  facilities  must  continue  to  be  main- 
tained and  used  for  the  few  hours  which  continue  to  be  flown  in  aircraft.  It 
seems  safe  to  say  that  the  military  training  cost  models  presently  employed  do 
not  provide  all  of  the  Information  needed  to  make  the  best  decisions  about  using 
simulators . 

The  second  reason  we  do  not  know  about  the  cost-effectiveness  of  simulator 
training  relates  to  the  effectiveness  question.  The  effectiveness  of  almost 
all  military  simulator  training  programs  is  being  assumed,  and,  in  many  cases, 
these  assumptions  may  be  in  error  by  significant  amounts.  There  has  been  vir- 
tually a total  absence  of  controlled  tests  designed  to  validate  military  simu- 
lator training  programs.  In  the  absence  of  such  tests,  simulator  training 
usually  has  been  substituted  for  aircraft  training  on  an  overkill  hnsis.  That 
is,  training  managers  have  substituted  simulator  training  for  aircraft  train- 
ing on  a ratio  that  typically  exceeds  parity,  thus  quite  likely  Inflating  simu- 
lator training  needs  and  costs.  The  consequences  of  excessive  simulator  train- 
ing may  be  better  training  overall,  of  course,  but  without  validating  tests, 
we  really  do  not  know  whether  the  ratios  are  adequate  or  excessive. 

CONCLUDING  COMMENT 

The  eight  problems  described  in  this  report  suggest  that  military  simulator 
training  systems  may  not  be  as  healthy  as  one  would  wish.  At  least  some  of 
these  problems  are  possibly  related  to  the  fact  that  simulator  training  is 
seldom  viewed  as  a system  that  encompasses,  but  Is  by  no  means  restricted  to, 
simulator  design,  testing  and  use.  Simulator  training  is  itself  part  of  a much 


11 


larger  training  system  that  includes  training  In  operational  vehicles  and 
classrooms  and  training  with  supporting  resources  such  as  airfields,  target 
ranges,  and  maneuver  areas.  As  long  as  we  treat  these  end  other  system  elements 
as  Independent,  with  responsibilities  fractionated,  simulator  training  will 
continue  to  be  beset  wich  problems  of  Importance  equal  to  or  greater  than 
those  identified  here. 


